Event Detection in Twitter Using Text and Image Fusion

In this paper, we describe an accurate and effective event detection method to detect events from Twitter stream. It detects events using visual information as well as textual information to improve the performance of the mining. It monitors Twitter stream to pick up tweets having texts and photos and stores them into database. Then it applies mining algorithm to detect the event. Firstly, it detects event based on text only by using the feature of the bag-of-words which is calculated using the term frequency-inverse document frequency (TF-IDF) method. Secondly, it detects the event based on image only by using visual features including histogram of oriented gradients (HOG) descriptors, grey-level co-occurrence matrix (GLCM), and color histogram. K nearest neighbours (Knn) classification is used in the detection. Finally, the final decision of the event detection is made based on the reliabilities of text only detection and image only detection. The experiment result showed that the proposed method achieved high accuracy of 0.93, comparing with 0.89 with texts only, and 0.86 with images only.